library(tidyverse)
demographics = read.csv('data/demos_anonymized.csv')
ids = read.csv('data/ids_anonymized.csv')
model_variables = read.csv('data/model_variables_anonymized.csv')

Introduction

Get a Quick Summary

It’s always a good idea to glance at the data

Grouping and Summarizing Operations

A very common operation is to do things by group(s) then create new summary variables.

Grouping and Summarizing Operations

Do multiple operations at once

Mapping functions to groups

We’ll use the map function to map the sum function to each element in the list

Grouping and Summarizing Operations

We can even do this with modeling and other operations.

This extracts the coefficients from a model run for each group.

Visualization

Visualizing data requires that you: - Consider carefully the information you want to display - And then how you want to display it

Tell a story with the data.

And have some fun with it!

ggplot2

The most widely used visualization package in R

Layers

Visualization can be thought of in a layered fashion

Start with the base, then build up

More pipes

ggplot2 uses a + as a pipe to add layers

We can pipe to any ggplot as we did before (%>%)

Aesthetics

Aesthetics (aes) map variables to visual properties

Geoms are the geometric units we want to display

Aesthetics

Stats

We can also use ggplot2 to create statistics we want to visualize

Typically used indirectly when geoms are called

Can be used for more direct control

Scales

Scales are used to add specifications to axes, colors, etc.

Facets

Facets allow another dimension to plots by group

Themes

Themes allow for customization

Two uses of a - a built-in versions (e.g. theme_minimal) - DIY (theme(…))

For the theme function, each argument, takes on a specific value or an element function:

Themes

Interactivity

Interactivity is a must-have tool for web-based presentation

Use to enhance exploration of the data - Not just because one can

Allows for additional dimensions

Even useful for exploring raw data

Interactivity

General

Interactivity

Specific functionality:

Plotly

traces - add_, work similar to geoms

modes - allow for points, lines, text and combinations

aesthetics - variables are denoted with ~, constants do not use - x =~ var1 vs x = 2

Plotly

Plotly uses the standard pipe %>%

Plotly

Plotly

Plotly

Use ggplotly to turn our formerly static plots into interactive ones.

Python examples

Visualization

matplotlib is the most common visualization module in Python, though it’s fairly dated at this point. As such we’ll use a ggplot implementation in python called plotnine.

Unfortunately for plotly, the interactivity makes it unusable within the R notebook (at present), so you may need to switch to Anaconda or other IDE to try other modules like plotly. Even Python users will still use R for easier visualization though, so feel free to do what you like there, then use ggplot etc. in R when the time comes.

That said, I’ll show a couple plots

LS0tCnRpdGxlOiAiTW9kdWxlIDI6IFN1bW1hcml6aW5nIGFuZCBWaXN1YWxpemluZyBEYXRhIgpvdXRwdXQ6IAogIGh0bWxfbm90ZWJvb2s6IAogICAgaGlnaGxpZ2h0OiBweWdtZW50cwogICAgdGhlbWU6IHNhbmRzdG9uZQogICAgY3NzOiBvdGhlci5jc3MKZWRpdG9yX29wdGlvbnM6IAogIGNodW5rX291dHB1dF90eXBlOiBpbmxpbmUKLS0tCgpgYGB7ciBpbml0LCBlY2hvPUZBTFNFfQojIHRoZXNlIG9wdGlvbnMgYXJlIHByaW1hcnkgdXNlZnVsIHRvIHRoZSBjcmVhdGlvbiBvZiB0aGUgaHRtbCBkb2N1bWVudAprbml0cjo6b3B0c19jaHVuayRzZXQoCiAgZWNobz1ULCAKICBldmFsID0gRiwKICBtZXNzYWdlID0gRiwgCiAgd2FybmluZyA9IEYsIAogIGNvbW1lbnQgPSBOQSwKICBSLm9wdGlvbnM9bGlzdCh3aWR0aD0xMjApLCAKICBjYWNoZS5yZWJ1aWxkPUYsIAogIGNhY2hlPUYsCiAgZmlnLmFsaWduPSdjZW50ZXInLCAKICBmaWcuYXNwID0gLjcsCiAgZGV2ID0gJ3N2ZycsIAogIGRldi5hcmdzPWxpc3QoYmcgPSAndHJhbnNwYXJlbnQnKQopCmBgYAoKYGBge3IgY2F0Y2h1cCwgZXZhbD1UUlVFfQpsaWJyYXJ5KHRpZHl2ZXJzZSkKZGVtb2dyYXBoaWNzID0gcmVhZC5jc3YoJ2RhdGEvZGVtb3NfYW5vbnltaXplZC5jc3YnKQppZHMgPSByZWFkLmNzdignZGF0YS9pZHNfYW5vbnltaXplZC5jc3YnKQptb2RlbF92YXJpYWJsZXMgPSByZWFkLmNzdignZGF0YS9tb2RlbF92YXJpYWJsZXNfYW5vbnltaXplZC5jc3YnKQpgYGAKCiMjIEludHJvZHVjdGlvbgoKIyMgR2V0IGEgUXVpY2sgU3VtbWFyeQoKSXQncyBhbHdheXMgYSBnb29kIGlkZWEgdG8gZ2xhbmNlIGF0IHRoZSBkYXRhCgpgYGB7ciBzdW1tYXJ5fQpnbGltcHNlKGRlbW9ncmFwaGljcykKYGBgCmBgYHtyfQpzdW1tYXJ5KGRlbW9ncmFwaGljc1ssMToxMF0pCmBgYAoKCgojIyBHcm91cGluZyBhbmQgU3VtbWFyaXppbmcgT3BlcmF0aW9ucwoKQSB2ZXJ5IGNvbW1vbiBvcGVyYXRpb24gaXMgdG8gZG8gdGhpbmdzIGJ5IGdyb3VwKHMpIHRoZW4gY3JlYXRlIG5ldyBzdW1tYXJ5IHZhcmlhYmxlcy4KCmBgYHtyIGdyb3VwX2J5LCBldmFsPVRSVUV9CmRlbW9ncmFwaGljcyAlPiUgCiAgZ3JvdXBfYnkobGlidXNlcikgJT4lIAogIHN1bW1hcmlzZShhZ2UgPSBtZWFuKGFnZSwgbmEucm0gPSBUKSkgCmBgYAoKCgojIyBHcm91cGluZyBhbmQgU3VtbWFyaXppbmcgT3BlcmF0aW9ucwoKRG8gbXVsdGlwbGUgb3BlcmF0aW9ucyBhdCBvbmNlCgpgYGB7ciBncm91cF9ieTIsIGV2YWw9VFJVRX0KZGVtb2dyYXBoaWNzICU+JSAKICBncm91cF9ieShsaWJ1c2VyKSAlPiUgCiAgc3VtbWFyaXNlKGFnZV9tZWFuID0gbWVhbihhZ2UsIG5hLnJtID0gVCksCiAgICAgICAgICAgIGFnZV9zZCA9IHNkKGFnZSwgbmEucm0gPSBUKSwKICAgICAgICAgICAgYWdlX21heCA9IG1heChhZ2UsIG5hLnJtID0gVCksCiAgICAgICAgICAgIHByb3BfbWFsZSA9IG1lYW4oZ2VuZGVyPT0nTWFsZScsIG5hLnJtID0gVCkpIApgYGAKCgoKIyMgIE1hcHBpbmcgZnVuY3Rpb25zIHRvIGdyb3VwcwoKV2XigJlsbCB1c2UgdGhlIGBtYXBgIGZ1bmN0aW9uIHRvICptYXAqIHRoZSBgc3VtYCBmdW5jdGlvbiB0byBlYWNoIGVsZW1lbnQgaW4gdGhlIGxpc3QKCmBgYHtyIG1hcCwgZXZhbD1UUlVFfQp4ID0gbGlzdCgxOjMsIDQ6NiwgNzo5KQptYXAoeCwgc3VtKQpgYGAKCgoKIyMgR3JvdXBpbmcgYW5kIFN1bW1hcml6aW5nIE9wZXJhdGlvbnMKCldlIGNhbiBldmVuIGRvIHRoaXMgd2l0aCBtb2RlbGluZyBhbmQgb3RoZXIgb3BlcmF0aW9ucy4gCgpUaGlzIGV4dHJhY3RzIHRoZSBjb2VmZmljaWVudHMgZnJvbSBhIG1vZGVsIHJ1biBmb3IgZWFjaCBncm91cC4KCmBgYHtyIGdyb3VwX2J5MywgZXZhbD1UUlVFfQpkZW1vZ3JhcGhpY3MgJT4lIAogIGRyb3BfbmEocmFjZSkgJT4lIAogIGdyb3VwX3NwbGl0KHJhY2UpICU+JSAgIAogIG1hcCh+bG0oYXdhcmRfdG90YWxfYW1vdW50IH4gZ2VuZGVyLCBkYXRhID0gLikpCmBgYAoKCiMjIFZpc3VhbGl6YXRpb24KClZpc3VhbGl6aW5nIGRhdGEgcmVxdWlyZXMgdGhhdCB5b3U6Ci0gQ29uc2lkZXIgY2FyZWZ1bGx5IHRoZSBpbmZvcm1hdGlvbiB5b3Ugd2FudCB0byBkaXNwbGF5Ci0gQW5kIHRoZW4gaG93IHlvdSB3YW50IHRvIGRpc3BsYXkgaXQKClRlbGwgYSBzdG9yeSB3aXRoIHRoZSBkYXRhLgoKQW5kIGhhdmUgc29tZSBmdW4gd2l0aCBpdCEKCgojIyBnZ3Bsb3QyCgpUaGUgbW9zdCB3aWRlbHkgdXNlZCB2aXN1YWxpemF0aW9uIHBhY2thZ2UgaW4gUgoKCgojIyBMYXllcnMKClZpc3VhbGl6YXRpb24gY2FuIGJlIHRob3VnaHQgb2YgaW4gYSBsYXllcmVkIGZhc2hpb24KClN0YXJ0IHdpdGggdGhlIGJhc2UsIHRoZW4gYnVpbGQgdXAKCiMjIE1vcmUgcGlwZXMKCmBnZ3Bsb3QyYCB1c2VzIGEgYCtgIGFzIGEgcGlwZSB0byBhZGQgbGF5ZXJzCgpgYGB7ciBnZ3Bsb3QyLCBldmFsPUZBTFNFfQpsaWJyYXJ5KGdncGxvdDIpCmdncGxvdChkYXRhKSArCiAgZ2VvbV9wb2ludChhZXMoeCA9IHZhcjEsIHkgPSB2YXIyKSkgKwogIGdlb21fbGluZSgpICsKICB0aGVtZShwbG90LmNhcHRpb24gPSBlbGVtZW50X3RleHQoc2l6ZSA9IDYpKQpgYGAKCldlIGNhbiBwaXBlICp0byogYW55IGdncGxvdCBhcyB3ZSBkaWQgYmVmb3JlIChgICU+JSBgKQoKCgojIyBBZXN0aGV0aWNzCgoqKkFlc3RoZXRpY3MqKiAoYGFlc2ApIG1hcCB2YXJpYWJsZXMgdG8gdmlzdWFsIHByb3BlcnRpZXMKCioqR2VvbXMqKiBhcmUgdGhlIGdlb21ldHJpYyB1bml0cyB3ZSB3YW50IHRvIGRpc3BsYXkKCgojIyBBZXN0aGV0aWNzCgpgYGB7ciBhZXN0aGV0aWNzfQptb2RlbF92YXJpYWJsZXMgJT4lIAogIGZpbHRlcihhd2FyZF90b3RhbF9hbW91bnQgPiAxZTcpICU+JSAKICBnZ3Bsb3QoKSArCiAgZ2VvbV9kZW5zaXR5KGFlcyh4ID0gYXdhcmRfdG90YWxfYW1vdW50LCAKICAgICAgICAgICAgICAgICAgIGNvbG9yID0gZmFjdG9yKGdlbmRlciksCiAgICAgICAgICAgICAgICAgICBmaWxsID0gZmFjdG9yKGdlbmRlcikpLAogICAgICAgICAgICAgICBhbHBoYSA9IC4yKSArIAogIHNjYWxlX3hfY29udGludW91cyhicmVha3MgPSAoMToxMCkgKiAxZTcsIHRyYW5zID0gJ2xvZycpCmBgYAoKYGBge3IgYWVzdGhldGljczJ9CmRlbW9ncmFwaGljcyAlPiUgCiAgZmlsdGVyKGF3YXJkX3llYXJfc3RhcnQgPCAyMDIwICYgYXdhcmRfeWVhcl9zdGFydCA+IDE5OTApICU+JSAgCiAgZ2dwbG90KGFlcyhhd2FyZF95ZWFyX3N0YXJ0LCBhd2FyZF90b3RhbF9sb2cpKSArIAogIGdlb21fc21vb3RoKGFlcyhjb2xvcj1mYWN0b3IobGlidXNlcikpKQpgYGAKCgojIyBTdGF0cwoKV2UgY2FuIGFsc28gdXNlIGdncGxvdDIgdG8gY3JlYXRlIHN0YXRpc3RpY3Mgd2Ugd2FudCB0byB2aXN1YWxpemUKClR5cGljYWxseSB1c2VkIGluZGlyZWN0bHkgd2hlbiBnZW9tcyBhcmUgY2FsbGVkCgpDYW4gYmUgdXNlZCBmb3IgbW9yZSBkaXJlY3QgY29udHJvbAoKYGBge3IgZ2dzdGF0czJ9CmdncGxvdChtb2RlbF92YXJpYWJsZXMsIGFlcyhhZ2UsIGF3YXJkX3RvdGFsX2Ftb3VudCkpICsKICBnZW9tX3BvaW50KGFscGhhID0gLjAyKSArCiAgc3RhdF9lbGxpcHNlKGNvbG9yID0gJyNmZjU1MDAnKQpgYGAKCgoKIyMgU2NhbGVzCgpTY2FsZXMgYXJlIHVzZWQgdG8gYWRkIHNwZWNpZmljYXRpb25zIHRvIGF4ZXMsIGNvbG9ycywgZXRjLgoKYGBge3Igc2NhbGVzfQptb2RlbF92YXJpYWJsZXMgJT4lIAogIGZpbHRlcihhd2FyZF90b3RhbF9hbW91bnQgPj0gMWU2KSAlPiUgCiAgZ2dwbG90KCkgKwogIGdlb21fZGVuc2l0eShhZXMoeCA9IGF3YXJkX3RvdGFsX2Ftb3VudCwgCiAgICAgICAgICAgICAgICAgICBjb2xvciA9IGdlbmRlciwKICAgICAgICAgICAgICAgICAgIGZpbGwgPSBnZW5kZXIpLAogICAgICAgICAgICAgICBhbHBoYSA9IC4yKSArIAogIHNjYWxlX3hfY29udGludW91cyhicmVha3MgPSBjKDFlNiwgNWU2LCAxZTcsIDIuNWU3LCA1ZTcpLCAKICAgICAgICAgICAgICAgICAgICAgdHJhbnMgPSAnbG9nJykgKwogIHNjYWxlX2ZpbGxfdmlyaWRpc19kKGJlZ2luID0gLjI1LCBlbmQgPSAuNSkgKwogIHNjYWxlX2NvbG9yX3ZpcmlkaXNfZChiZWdpbiA9IC4yNSwgZW5kID0gLjUpIApgYGAKCgojIyBGYWNldHMKCkZhY2V0cyBhbGxvdyBhbm90aGVyIGRpbWVuc2lvbiB0byBwbG90cyBieSBncm91cAoKLSAqKmZhY2V0X2dyaWQqKjogcmV0dXJucyBhIG1hdHJpeCBvZiBsaWtlIGRpbWVuc2lvbnMKLSAqKmZhY2V0X3dyYXAqKjogbW9yZSBmbGV4aWJsZSBzcGVjaWZpY2F0aW9uCgpgYGB7ciBmYWNldHN9Cm1vZGVsX3ZhcmlhYmxlcyAlPiUgCiAgZ2dwbG90KCkgKwogIGdlb21fZGVuc2l0eShhZXMoeCA9IGF3YXJkX3RvdGFsX2Ftb3VudCwgCiAgICAgICAgICAgICAgICAgICBjb2xvciA9IGxpYnVzZXIsCiAgICAgICAgICAgICAgICAgICBmaWxsID0gbGlidXNlciksCiAgICAgICAgICAgICAgIGFscGhhID0gLjIpICsKICBmYWNldF93cmFwKH5nZW5kZXIpCmBgYAoKCgoKIyMgVGhlbWVzCgpgVGhlbWVzYCBhbGxvdyBmb3IgY3VzdG9taXphdGlvbgoKVHdvIHVzZXMgb2YgYSAKLSBhIGJ1aWx0LWluIHZlcnNpb25zIChlLmcuIGB0aGVtZV9taW5pbWFsYCkKLSBESVkgIChgdGhlbWVgKC4uLikpCgpGb3IgdGhlIHRoZW1lIGZ1bmN0aW9uLCBlYWNoIGFyZ3VtZW50LCB0YWtlcyBvbiBhIHNwZWNpZmljIHZhbHVlIG9yIGFuIGVsZW1lbnQgZnVuY3Rpb246CgotIGBlbGVtZW50X3JlY3RgCi0gYGVsZW1lbnRfbGluZWAKLSBgZWxlbWVudF90ZXh0YAotIGBlbGVtZW50X2JsYW5rYAoKCiMjIFRoZW1lcwoKYGBge3IgdGhlbWVzMX0KbW9kZWxfdmFyaWFibGVzICU+JSAKICBnZ3Bsb3QoKSArCiAgZ2VvbV9zbW9vdGgoYWVzKHggPSBhZ2UsCiAgICAgICAgICAgICAgICB5ID0gYXdhcmRfdG90YWxfYW1vdW50LAogICAgICAgICAgICAgICAgY29sb3IgPSBsaWJ1c2VyKSwKICAgICAgICAgICAgICAgYWxwaGEgPSAuMikgKyAKICB0aGVtZV9taW5pbWFsKCkKYGBgCgoKYGBge3IgdGhlbWVzMn0KbW9kZWxfdmFyaWFibGVzICU+JSAKICBnZ3Bsb3QoKSArCiAgZ2VvbV9zbW9vdGgoYWVzKHggPSBhZ2UsCiAgICAgICAgICAgICAgICB5ID0gYXdhcmRfdG90YWxfYW1vdW50LAogICAgICAgICAgICAgICAgY29sb3IgPSBsaWJ1c2VyKSwKICAgICAgICAgICAgICAgYWxwaGEgPSAuMikgKyAKICB0aGVtZShheGlzLnRleHQueCA9IGVsZW1lbnRfdGV4dChzaXplPTEyKSwKICAgICAgICBwYW5lbC5ncmlkLm1pbm9yLnggPSBlbGVtZW50X2JsYW5rKCksCiAgICAgICAgcGxvdC5iYWNrZ3JvdW5kID0gZWxlbWVudF9yZWN0KGNvbG9yID0gJ3Jvc3licm93bicpLAogICAgICAgIHBhbmVsLmJhY2tncm91bmQgPSBlbGVtZW50X3JlY3QoZmlsbCA9ICdwYXBheWF3aGlwJykpCmBgYAoKIyMgSW50ZXJhY3Rpdml0eQoKSW50ZXJhY3Rpdml0eSBpcyBhIG11c3QtaGF2ZSB0b29sIGZvciB3ZWItYmFzZWQgcHJlc2VudGF0aW9uCgpVc2UgdG8gZW5oYW5jZSBleHBsb3JhdGlvbiBvZiB0aGUgZGF0YQotIE5vdCBqdXN0IGJlY2F1c2Ugb25lIGNhbgoKQWxsb3dzIGZvciBhZGRpdGlvbmFsIGRpbWVuc2lvbnMKCkV2ZW4gdXNlZnVsIGZvciBleHBsb3JpbmcgcmF3IGRhdGEKCgoKIyMgSW50ZXJhY3Rpdml0eQoKCkdlbmVyYWwgCgotIFtwbG90bHldKGh0dHBzOi8vcGxvdC5seS9yLykKICAtIGFsc28gdXNlZCBpbiBQeXRob24sIE1hdGxhYiwgSnVsaWEKICAtIGNhbiBjb252ZXJ0IGBnZ3Bsb3QyYCBpbWFnZXMgdG8gaW50ZXJhY3RpdmUgb25lcwogIAotIFtoaWdoY2hhcnRlcl0oaHR0cDovL2prdW5zdC5jb20vaGlnaGNoYXJ0ZXIvKQogIC0gZ2VuZXJhbCB3cmFwcGVyIGZvciBoaWdoY2hhcnRzLmpzIAogIC0gd29ya3Mgd2l0aCBzb21lIFIgcGFja2FnZXMgb3V0IG9mIHRoZSBib3gKICAKLSBbcmJva2VoXShodHRwOi8vaGFmZW4uZ2l0aHViLmlvL3Jib2tlaC8pCiAgLSBsaWtlIHBsb3RseSwgaXQgYWxzbyBoYXMgY3Jvc3MgbGFuZ3VhZ2Ugc3VwcG9ydAoKCgojIyBJbnRlcmFjdGl2aXR5CgpTcGVjaWZpYyBmdW5jdGlvbmFsaXR5OgoKLSBbRFRdKGh0dHBzOi8vcnN0dWRpby5naXRodWIuaW8vRFQvKQogIC0gaW50ZXJhY3RpdmUgZGF0YSB0YWJsZXMKICAKLSBbbGVhZmxldF0oaHR0cHM6Ly9yc3R1ZGlvLmdpdGh1Yi5pby9sZWFmbGV0LykKICAgIC0gbWFwcyB3aXRoIE9wZW5TdHJlZXRNYXAKICAgIAotIFt2aXNOZXR3b3JrXShodHRwOi8vZGF0YXN0b3JtLW9wZW4uZ2l0aHViLmlvL3Zpc05ldHdvcmsvKQogICAgLSBOZXR3b3JrIHZpc3VhbGl6YXRpb24KICAgIAoKCiMjIFBsb3RseQoKYHRyYWNlc2AKLSBgYWRkX2AsIHdvcmsgc2ltaWxhciB0byBnZW9tcwogIApgbW9kZXNgCi0gYWxsb3cgZm9yIHBvaW50cywgbGluZXMsIHRleHQgYW5kIGNvbWJpbmF0aW9ucwoKYGFlc3RoZXRpY3NgCi0gdmFyaWFibGVzIGFyZSBkZW5vdGVkIHdpdGggYH5gLCBjb25zdGFudHMgZG8gbm90IHVzZQotIGB4ID1+IHZhcjFgIHZzIGB4ID0gMmAKCgoKCiMjIFBsb3RseQoKUGxvdGx5IHVzZXMgdGhlIHN0YW5kYXJkIHBpcGUgYCU+JWAKCmBgYHtyIHBsb3RseX0KbGlicmFyeShwbG90bHkpCgptb2RlbF92YXJpYWJsZXMgJT4lIAogIHBsb3RfbHkoeCA9IH5nZW5kZXIsIHkgPSB+IGFnZSkgJT4lIAogIGFkZF9ib3hwbG90KGNvbG9yID1+IGdlbmRlcikKYGBgCgoKCiMjIFBsb3RseQoKYGBge3IgcGxvdGx5MX0KbGlicmFyeShwbG90bHkpCgptb2RlbF92YXJpYWJsZXMgJT4lIAogIGdyb3VwX2J5KGxpYnVzZXIsIGdlbmRlcikgJT4lIAogIHN1bW1hcmlzZShhd2FyZCA9IG1lYW4oYXdhcmRfdG90YWxfYW1vdW50KSkgJT4lIAogIHBsb3RfbHkoeCA9IH5saWJ1c2VyLCAKICAgICAgICAgIHkgPSB+YXdhcmQsIAogICAgICAgICAgY29sb3IgPSB+Z2VuZGVyLAogICAgICAgICAgdGV4dCA9IH5yb3VuZChhd2FyZCksIAogICAgICAgICAgdGV4dHBvc2l0aW9uID0gJ2F1dG8nLCAKICAgICAgICAgIHR5cGUgPSAnYmFyJykgJT4lIAogIGxheW91dChiYXJnYXAgPSAwLjI1LCAKICAgICAgICAgYmFyZ3JvdXBnYXAgPSAwLjI1KQpgYGAKCgoKIyMgUGxvdGx5CgpgYGB7ciBwbG90bHkyfQppbml0ID0gZ2xtKGF3YXJkX3RvdGFsX2Ftb3VudCA+PSA1MDAwMDAwIH4gYWdlKmxpYnVzZXIsIAogICAgICAgICAgIGRhdGEgPSBtb2RlbF92YXJpYWJsZXMsIAogICAgICAgICAgIGZhbWlseSA9IGJpbm9taWFsKQoKbW9kZWxfdmFyaWFibGVzICU+JSAKICBtb2RlbHI6OmFkZF9wcmVkaWN0aW9ucyhpbml0LCB0eXBlID0gJ3Jlc3BvbnNlJykgJT4lIAogIHBsb3RfbHkoeCA9IH5hZ2UsIHkgPSB+IHByZWQpICU+JSAKICBhZGRfbGluZXMoY29sb3IgPX4gbGlidXNlciwgbGluZSA9IGxpc3Qoc2hhcGUgPSAic3BsaW5lIikpICU+JSAKICBsYXlvdXQodGl0bGUgPSAnUHJlZGljdGVkIFByb2IuIEF3YXJkID4gMSBtaWwnKQpgYGAKCgoKIyMgUGxvdGx5CgpVc2UgZ2dwbG90bHkgdG8gdHVybiBvdXIgZm9ybWVybHkgc3RhdGljIHBsb3RzIGludG8gaW50ZXJhY3RpdmUgb25lcy4gCgpgYGB7ciBwbG90bHkzfQpwID0gbW9kZWxfdmFyaWFibGVzICU+JSAKICBnZ3Bsb3QoKSArCiAgZ2VvbV9kZW5zaXR5KGFlcyh4ID0gbG9nKGF3YXJkX3RvdGFsX2Ftb3VudCksIAogICAgICAgICAgICAgICAgICAgY29sb3IgPSBsaWJ1c2VyLAogICAgICAgICAgICAgICAgICAgZmlsbCA9IGxpYnVzZXIpLAogICAgICAgICAgICAgICBhbHBoYSA9IC4yKSArIAogIGZhY2V0X3dyYXAofiBnZW5kZXIpIApnZ3Bsb3RseSgpCmBgYAoKCgoKIyMgUHl0aG9uIGV4YW1wbGVzCgojIyMgSW5pdAoKYGBge3B5dGhvbiBweV9pbml0LCBlbmdpbmUucGF0aD0gJy9Vc2Vycy9taWNsL2FuYWNvbmRhMy9iaW4vcHl0aG9uJ30KaW1wb3J0IHBhbmRhcyBhcyBwZAppbXBvcnQgbnVtcHkgYXMgbnAKCmRlbW9ncmFwaGljcyA9IHBkLnJlYWRfY3N2KCdkYXRhL2RlbW9zX2Fub255bWl6ZWQuY3N2JykKaWRzID0gcGQucmVhZF9jc3YoJ2RhdGEvaWRzX2Fub255bWl6ZWQuY3N2JykKbW9kZWxfdmFyaWFibGVzID0gcGQucmVhZF9jc3YoJ2RhdGEvbW9kZWxfdmFyaWFibGVzX2Fub255bWl6ZWQuY3N2JykKYGBgCgoKIyMjICBHcm91cGluZyBhbmQgU3VtbWFyaXppbmcgZGF0YQoKYGBge3B5dGhvbiBweV9zdW1tYXJ5fQpkZW1vZ3JhcGhpY3MuZGVzY3JpYmUoaW5jbHVkZSA9ICdhbGwnKQpgYGAKYGBge3B5dGhvbiBweV9zdW1tYXJ5X251bWVyaWN9CmRlbW9ncmFwaGljcy5kZXNjcmliZShpbmNsdWRlID0gW25wLm51bWJlcl0pCmBgYAoKYGBge3B5dGhvbiBweV9zdW1tYXJ5X3N0cmluZ30KZGVtb2dyYXBoaWNzLmRlc2NyaWJlKGluY2x1ZGUgPSBbbnAub2JqZWN0XSkKYGBgCgoKCgpgYGB7cHl0aG9uIHB5X2dyb3VwX2J5LCBldmFsPVRSVUV9CmxpYl9ncm91cCA9IGRlbW9ncmFwaGljcy5ncm91cGJ5KCdsaWJ1c2VyJywgc29ydD1UcnVlLCApCgojIGF1dG9tYXRpY2FsbHkgY2hvb3NlcyBudW1lcmljCmxpYl9ncm91cC5tZWFuKCkKYGBgCgoKYGBge3B5dGhvbiBweV9ncm91cF9ieTIsIGV2YWw9VFJVRX0KbGliX2dyb3VwLmdldF9ncm91cCgwKS5oZWFkKCkKYGBgCgoKYGBge3B5dGhvbiBweV9ncm91cF9ieTMsIGV2YWw9VFJVRX0KbGliX2dyb3VwLnNpemUoKQpgYGAKCmBgYHtweXRob24gcHlfZ3JvdXBfYnk0LCBldmFsPVRSVUV9CmxpYl9ncm91cC5kZXNjcmliZSgpCmBgYAojIyMgTWFwcGluZyBhIGZ1bmN0aW9uCgpgYGB7cHl0aG9uIHB5X21hcH0KeCA9IFtbMSwyLDNdLCBbNCw1LDZdLCBbNyw4LDldXQpsaXN0KG1hcChucC5zdW0sIHgpKQpgYGAKCmBgYHtweXRob24gcHlfbWFwMn0KZGVtb2dyYXBoaWNzLnNlbGVjdF9kdHlwZXMoJ251bWJlcicpLmFwcGx5KG5wLm1lYW4pCmBgYAoKCiMjIyBWaXN1YWxpemF0aW9uCgpgbWF0cGxvdGxpYmAgaXMgdGhlIG1vc3QgY29tbW9uIHZpc3VhbGl6YXRpb24gbW9kdWxlIGluIFB5dGhvbiwgdGhvdWdoIGl0J3MgZmFpcmx5IGRhdGVkIGF0IHRoaXMgcG9pbnQuIEFzIHN1Y2ggd2UnbGwgdXNlIGEgZ2dwbG90IGltcGxlbWVudGF0aW9uIGluIHB5dGhvbiBjYWxsZWQgYHBsb3RuaW5lYC4KClVuZm9ydHVuYXRlbHkgZm9yIHBsb3RseSwgdGhlIGludGVyYWN0aXZpdHkgbWFrZXMgaXQgdW51c2FibGUgd2l0aGluIHRoZSBSIG5vdGVib29rIChhdCBwcmVzZW50KSwgc28geW91IG1heSBuZWVkIHRvIHN3aXRjaCB0byBBbmFjb25kYSBvciBvdGhlciBJREUgdG8gdHJ5IG90aGVyIG1vZHVsZXMgbGlrZSBgcGxvdGx5YC4gIEV2ZW4gUHl0aG9uIHVzZXJzIHdpbGwgc3RpbGwgdXNlIFIgZm9yIGVhc2llciB2aXN1YWxpemF0aW9uIHRob3VnaCwgc28gZmVlbCBmcmVlIHRvIGRvIHdoYXQgeW91IGxpa2UgdGhlcmUsIHRoZW4gdXNlIGdncGxvdCBldGMuIGluIFIgd2hlbiB0aGUgdGltZSBjb21lcy4KClRoYXQgc2FpZCwgSSdsbCBzaG93IGEgY291cGxlIHBsb3RzCgpgYGB7cHl0aG9uIHBsb3RuaW5lfQppbXBvcnQgcGxvdG5pbmUKYGBgCgpgYGB7cHl0aG9uIGdnMX0KZHBsb3QgPSBkZW1vZ3JhcGhpY3NbKGRlbW9ncmFwaGljcy5hd2FyZF95ZWFyX3N0YXJ0IDwgMjAyMCkgJiAoZGVtb2dyYXBoaWNzLmF3YXJkX3llYXJfc3RhcnQgPiAxOTkwKSAmIChkZW1vZ3JhcGhpY3MuYXdhcmRfdG90YWxfbG9nID4gMTEpXQpkcGxvdApnZ3Bsb3QoZHBsb3QsIGFlcyh4PSdhd2FyZF95ZWFyX3N0YXJ0JywgeT0nYXdhcmRfdG90YWxfbG9nJykpICsgZ2VvbV9zbW9vdGgoYWVzKGdyb3VwID0gJ2xpYnVzZXInLCBjb2xvciA9ICdsaWJ1c2VyJykpCmBgYAoKCmBgYHtweXRob24gZ2cyfQp5ZWFyX2F3YXJkX2F2ZXJhZ2UgPSBkZW1vZ3JhcGhpY3NbZGVtb2dyYXBoaWNzLmxpYnVzZXIgPT0gMV0uZ3JvdXBieShbJ2F3YXJkX2hvbWVfZGVwdCcsICdhd2FyZF95ZWFyX3N0YXJ0J10pLm1lYW4oKQp5ZWFyX2F3YXJkX2F2ZXJhZ2UKYGBgCgoKCmBgYHtweXRob24gcHlfcGxvdGx5X2luaXR9CmltcG9ydCBwbG90bHkKaW1wb3J0IHBsb3RseS5wbG90bHkgYXMgcHkKZnJvbSBwbG90bHkub2ZmbGluZSBpbXBvcnQgaW5pdF9ub3RlYm9va19tb2RlCmltcG9ydCBwbG90bHkuZ3JhcGhfb2JqcyBhcyBnbwpwbG90bHkub2ZmbGluZS5pbml0X25vdGVib29rX21vZGUoY29ubmVjdGVkPVRydWUpCmBgYAoKCmBgYHtweXRob24gcHlfcGxvdGx5fQp5MCA9IG5wLnJhbmRvbS5yYW5kbig1MCktMQp5MSA9IG5wLnJhbmRvbS5yYW5kbig1MCkrMQoKdHJhY2UwID0gZ28uQm94KAogICAgeT15MAopCnRyYWNlMSA9IGdvLkJveCgKICAgIHk9eTEKKQpkYXRhID0gW3RyYWNlMCwgdHJhY2UxXQpweS5pcGxvdChkYXRhKQpgYGAKCgpgYGB7cHl0aG9ufQpgYGAK